Generalization in Threshold Networks, Combined Decision Trees and Combined Mask Perceptrons

We derive an upper bound on the generalization error of classi ers from a certain class of threshold networks. The bound depends on the margin of the classi er and the average complexity of the hidden units (where the average is over the weights assigned to each hidden unit). By representing convex combinations of decision trees or mask perceptrons as such threshold networks we obtain similar bounds on the generalization error of these classi ers. These bounds have immediate application to combinations of decision trees or mask perceptrons by majority vote which appear in techniques such as boosting, bagging and arcing. For combined decision trees, previous bounds depend on either the complexity of the most complex decision tree in the combination or the average complexity of the individual decision trees, where the complexity of each decision tree depends on the total number of leaves in the tree. The bound in this paper depends on the average complexity of the individual decision trees, where the complexity of each decision tree depends on the e ective number of leaves, a quantity which can be signi cantly smaller than the total number of leaves.